Asymptotic sequential Rademacher complexity of a finite function class

For a finite function class we describe the large sample limit of the sequential Rademacher complexity in terms of the viscosity solution of a G-heat equation. In the language of Peng’s sublinear expectation theory, the same quantity equals to the expected value of the largest order statistics of a multidimensional G-normal random variable. We illustrate this result by deriving upper and lower bounds for the asymptotic sequential Rademacher complexity. 1. Preliminaries The notion of sequential Rademacher complexity was introduced in [10] (see also [11, 12]). Let (εi) n i=1 be independent Rademacher random variables: P(εi = 1) = P(εi = −1) = 1/2. Consider a set Z, endowed with a σ-algebra G , and a collection F of Borel measurable functions f : Z 7→ R. For any sequence of functions zn : {−1, 1}n−1 7→ Z, n ≥ 1, where z1 is simply an element of Z, put Rn(F , z 1 ) = 1 √ n E sup f∈F n ∑ t=1 εtf(zt(ε t−1 1 )). By a1 we denote a sequence (a1, . . . , an). The sequential Rademacher complexity of the function class F is defined by Rn(F) = sup z 1 Rn(F , z 1 ). (1.1) The incentives to study this quantity come from the online learning theory, where on every round t a learner picks an element qt from the setQ of all probability distributions defined on the Borel σ-algebra of the metric space F , and an adversary picks an element zt ∈ Z. The value ∫ F f(zt) qt(df) determines the loss of the learner. The normalized cumulative regret over n rounds is defined by Rn(q n 1 , z n 1 ) = 1 √ n ( n ∑